Although this invention is being disclosed in connection with cervical cancer, it is applicable to many other areas of medicine. Cervical cancer is preventable with early detection but still comprises approximately 12% of all cancer cases in women worldwide (World Health Organization, “Global Health Risks”, December 2009, incorporated herein by reference). This considerable number of cervical cancer cases is mainly attributed to the lack of cervical cancer prevention programs in developing countries. Even though cervical cancer prevention programs such as the Papanicolaou (Pap) smear have been effective in reducing the incidence and mortality of cervical cancer, developing countries often lack the sophisticated laboratory equipment, highly trained personnel and financial resources necessary to implement these programs (R. Sankaranarayanan, A. M. Budukh, and R. Rajkumar, “Effective screening programmes for cervical cancer in low- and middle-income developing countries,” Bulletin of the World Health Organization 79, pp. 954-962, 2001; H. S. Cronje, “Screening for cervical cancer in developing countries,” International Journal of Gynecology and Obstetrics 84(2), pp. 101-108, 2004; and A. Batson, F. Meheus, and S. Brooke, “Chapter 26: Innovative financing mechanisms to accelerate the introduction of HPV vaccines in developing countries,” Vaccine 24, pp. 219-225, 2006; incorporated herein by reference). Without a cost effective cervical cancer screening solution, cervical cancer remains a leading cause of cancer-related death among women in developing countries.
To address this problem, alternative cost effective cervical cancer screening methods have been investigated (L. Denny, L. Kuhn, A. Pollack, H. Wainwright, and T. Wright, “Evaluation of alternative methods of cervical cancer screening for resource-poor settings,” Cancer 89(4), pp. 826-833, 2000; T. C. Wright Jr, M. Menton, J. F. Myrtle, C. Chow, and A. Singer, “Visualization techniques (colposcopy, direct visual inspection, and spectroscopic and other visual methods). Summary of task force 7,” Acta Cytologica 46(5), pp. 793-800, 2002; J. Benavides, S. Chang, S. Park, R. Richards-Kortum, N. MacKinnon, C. MacAulay, A. Milbourne, A. Malpica, and M. Follen, “Multispectral digital colposcopy for in vivo detection of cervical cancer,” Optics Express 11(10), pp. 1223-1236, 2003; S. J. Goldie, L. Gaffikin, J. D. Goldhaber-Fiebert, A. Gordillo-Tobar, C. Levin, C. Mahe, and T. C. Wright, “Cost-effectiveness of cervical-cancer screening in five developing countries,” The New England Journal of Medicine 353(20), p. 2158, 2005; J. Jeronimo, O. Morales, J. Horna, J. Pariona, J. Manrique, J. Rubi?s, and R. Takahashi, “Visual inspection with acetic acid for cervical cancer screening outside of low-resource settings,” Revista panamericana de salud publica 17, pp. 1-5, 2005; D. Roblyer, S. Y. Park, R. Richards-Kortum, I. Adewole, and M. Follen, “Objective screening for cervical cancer in developing nations: Lessons from Nigeria,” Gynecologic Oncology 107(1S), pp. 94-97, 2007; S. Y. Park, M. Follen, A. Milbourne, H. Rhodes, A. Malpica, N. MacKinnon, C. MacAulay, M. K. Markey, and R. Richards-Kortum, “Automated image analysis of digital colposcopy for the detection of cervical neoplasia,” Journal of Biomedical Optics 13, p. 014029, 2008; and N. Thekkek and R. Richards-Kortum, “Optical imaging for cervical cancer detection: solutions for a continuing global problem,” Nature Reviews. Cancer 8(9), p. 725, 2008, incorporated herein by reference) and considerable efforts have been devoted to digital colposcopy with automated image analysis techniques (W. E. Crisp, B. L. Craine, and E. A. Craine, “The computerized digital imaging colposcope: future directions,” American Journal of Obstetrics and Gynecology 162(6), p. 1491, 1990; B. L. Craine and E. R. Craine, “Digital imaging colposcopy: basic concepts and applications,” Obstetrics & Gynecology 82(5), p. 869, 1993; M. I. Shafi, J. A. Dunn, R. Chenoy, E. J. Buxton, C. Williams, and D. M. Luesley, “Digital imaging colposcopy, image analysis and quantification of the colposcopic image,” British Journal of Obstetrics and Gynaecology 101(3), p. 234, 1994; P. M. Cristoforoni, D. Gerbaldo, A. Perino, R. Piccoli, F. J. Montz, and G. L. Capitanio, “Computerized colposcopy: Results of a pilot study and analysis of its clinical relevance,” Obstetrics and Gynecology 85, p. 1011, 1995; Q. Ji, J. Engel, and E. Craine, “Texture analysis for classification of cervix lesions,” IEEE Transactions on Medical Imaging 19(11), pp. 1144-1149, 2000; E. D. Dickman, T. J. Doll, C. K. Chiu, and D. G. Ferris, “Identification of cervical neoplasia using a simulation of human vision,” Journal of Lower Genital Tract Disease 5(3), p. 144, 2001; S. Gordon, G. Zimmerman, and H. Greenspan, “Image segmentation of uterine cervix images for indexing in PACS,” Proc. of 17th IEEE Symposium on Computer-Based Medical Systems, pp. 298-303, 2004; A. Milbourne, S. Y. Park, J. L. Benedet, D. Miller, T. Ehlen, H. Rhodes, A. Malpica, J. Matisic, D. Van Niekirk, and E. N. Atkinson; “Results of a pilot study of multispectral digital colposcopy for the in vivo detection of cervical intraepithelial neoplasia,” Gynecologic Oncology 99(3S), pp. 67-75, 2005; S. Gordon, G. Zimmerman, R. Long, S. Antani, J. Jeronimo, and H. Greenspan, “Content analysis of uterine cervix images: initial steps towards content based indexing and retrieval of cervigrams,” Proc. of SPIE Medical Imaging 6144, pp. 1549-1556, 2006; W. Li and A. Poirson, “Detection and characterization of abnormal vascular patterns in automated cervical image analysis,” Lecture Notes in Computer Science 4292, p. 627, 2006; W. Li, J. Gu, D. Ferris, and A. Poirson, “Automated image analysis of uterine cervical images,” Proc. of SPIE Medical Imaging 6514. pp. 65142P-1 (2007); S. Y. Park, “A study on diagnostic image analysis for the detection of precancerous lesions using multispectral digital images,” PhD Thesis University of Texas at Austin, 2007; S. Y. Park, M. Follen, A. Milbourne, H. Rhodes, A. Malpica, N. MacKinnon, C. MacAulay, M. K. Markey, and R. Richards-Kortum, “Automated image analysis of digital colposcopy for the detection of cervical neoplasia,” Journal of Biomedical Optics 13, p. 014029, 2008; W. Li, S. Venkataraman, U. Gustafsson, J. C. Oyama, D. G. Ferris, and R. W. Lieberman, “Using acetowhite opacity index for detecting cervical intraepithelial neoplasia,” Journal of Biomedical Optics, vol. 14, p. 014020, 2009; and H. G. Acosta-Mesa, N. Cruz-Ramirez and R. Hermandez-Jimenez, “Aceto-white temporal pattern classification using k-NN to identify precancerous cervical lesion in colposcopic images,” Computers in Biology and Medicine, 39(9), pp. 778-784, 2009, incorporated herein with reference).
Many studies have shown that digital colposcopy with image-based diagnosis of cancer and pre-cancer has the potential to improve, or even replace, conventional colposcopy. The consistent and accurate diagnoses provided by digital image analysis have the potential to allow less experienced physicians to provide a standard of care on par with expert colposcopists. In the early 1990s, several studies showed the feasibility of using digital image processing techniques to automatically interpret colposcopic images (W. E. Crisp, B. L. Craine, and E. A. Craine, “The computerized digital imaging colposcope: future directions,” American Journal of Obstetrics and Gynecology 162(6), p. 1491, 1990; B. L. Craine and E. R. Craine, “Digital imaging colposcopy: basic concepts and applications,” Obstetrics & Gynecology 82(5), p. 869, 1993; M. I. Shafi, J. A. Dunn, R. Chenoy, E. J. Buxton, C. Williams, and D. M. Luesley, “Digital imaging colposcopy, image analysis and quantification of the colposcopic image,” British Journal of Obstetrics and Gynaecology 101(3), p. 234, 1994; and P. M. Cristoforoni, D. Gerbaldo, A. Perino, R. Piccoli, F. J. Montz, and G. L. Capitanio, “Computerized colposcopy: Results of a pilot study and analysis of its clinical relevance,” Obstetrics and Gynecology 85, p. 1011, 1995, incorporated herein by reference). In these early studies, diagnostic image interpretation relied primarily on qualitative image assessment from expert colposcopists and provided limited quantitative analysis.
Since these early proof-of-principle reports, automated algorithms have been designed with the goal of minimizing the need for provider (physician) intervention (E. D. Dickman, T. J. Doll, C. K. Chiu, and D. G. Ferris, “Identification of cervical neoplasia using a simulation of human vision,”Journal of Lower Genital Tract Disease 5(3), p. 144, 2001; S. Gordon, G. Zimmerman, and H. Greenspan, “Image segmentation of uterine cervix images for indexing in PACS,” Proc. of 17th IEEE Symposium on Computer-Based Medical Systems, pp. 298-303, 2004; A. Milbourne, S. Y. Park, J. L. Benedet, D. Miller, T. Ehlen, H. Rhodes, A. Malpica, J. Matisic, D. Van Niekirk, and E. N. Atkinson, “Results of a pilot study of multispectral digital colposcopy for the in vivo detection of cervical intraepithelial neoplasia,” Gynecologic Oncology 99(3S), pp. 67-75, 2005; S. Gordon, G. Zimmerman, R. Long, S. Antani, J. Jeronimo, and H. Greenspan, “Content analysis of uterine cervix images: initial steps towards content based indexing and retrieval of cervigrams,” Proc. of SPIE Medical Imaging 6144, pp. 1549-1556, 2006; W. Li, J. Gu, D. Ferris, and A. Poirson, “Automated image analysis of uterine cervical images,” Proc. of SPIE 6514, pp. 65142P-, 2007; S. Y. Park, “A study on diagnostic image analysis for the detection of precancerous lesions using multispectral digital images,” PhD Thesis University of Texas at Austin, 2007; S. Y. Park, M. Follen, A. Milbourne, H. Rhodes, A. Malpica, N. MacKinnon, C. MacAulay, M. K. Markey, and R. Richards-Kortum, “Automated image analysis of digital colposcopy for the detection of cervical neoplasia,” Journal of Biomedical Optics 13, p. 014029, 2008; W. Li, S. Venkataraman, U. Gustafsson, J. C. Oyama, D. G. Ferris, and R. W. Lieberman, “Using acetowhite opacity index for detecting cervical intraepithelial neoplasia,” Journal of Biomedical Optics 14, p. 014020, 2009; H. G. Acosta-Mesa, N. Cruz-Ramirez and R. Hermandez-Jimenez, “Aceto-white temporal pattern classification using k-NN to identify precancerous cervical lesion in colposcopic images,” Computers in Biology and Medicine, 39(9), pp. 778-784, 2009; W. Li, R. W. Lieberman, S. Nie, Y. Xie, M. Eldred, and J. Oyama, “Histopathology reconstruction on digital imagery,” Proc. of SPIE Medical Imaging 7263, p. 726303, 2009, incorporated herein by reference).
Dickman et al. (E. D. Dickman, T. J. Doll, C. K. Chiu, and D. G. Ferris, “Identification of cervical neoplasia using a simulation of human vision,” Journal of Lower Genital Tract Disease 5(3), p. 144, 2001, incorporated herein by reference) investigated the detection of cervical cancer and precursors from cervix images using a computer simulation of human vision. They trained a vision system to recognize normal and abnormal cervical images, and demonstrated 100% sensitivity and 98% specificity in detecting CIN3 on a very small data set of 8 images only.
Gordon and Li (S. Gordon, G. Zimmerman, and H. Greenspan, “Image segmentation of uterine cervix images for indexing in PACS,” Proc. of 17th IEEE Symposium on Computer-Based Medical Systems, pp. 298-303, 2004; S. Gordon, G. Zimmerman, R. Long, S. Antani, J. Jeronimo, and H. Greenspan, “Content analysis of uterine cervix images: initial steps towards content based indexing and retrieval of cervigrams,” Proc. of SPIE Medical Imaging 6144, pp. 1549-1556, 2006; and W. Li, J. Gu, D. Ferris, and A. Poirson, “Automated image analysis of uterine cervical images,” Proc. of SPIE Medical Imaging 6514, p. 65142P-1, 2007, incorporated herein by reference) developed image analysis algorithms to segment the anatomical regions of the cervix, such as the columnar epithelium, the squamous epithelium, the endo-cervical canal, and the transformation zone, based on color intensity values. Their research showed that a potential for accurate segmentation of the cervical anatomy exists. However, their work did not incorporate spatial relationships between tissue types and other diagnostic features, and they did not report the diagnostic accuracy of their algorithm.
Li et al. (W. Li, S. Venkataraman, U. Gustafsson, J. C. Oyama, D. G. Ferris, and R. W. Lieberman, “Using acetowhite opacity index for detecting cervical intraepithelial neoplasia,” Journal of Biomedical Optics 14, p. 014020, 2009, incorporated herein by reference) designed a computer-aided diagnostic system using an acetowhitening opacity index with a reported patient-based diagnostic result of 88% sensitivity and 84% specificity.
Similarly, Park et al (S. Y. Park, “A study on diagnostic image analysis for the detection of precancerous lesions using multispectral digital images,” PhD Thesis University of Texas at Austin, 2007; and S. Y. Park, M. Follen, A. Milbourne, H. Rhodes, A. Malpica, N. MacKinnon, C. MacAulay, M. K. Markey, and R. Richards-Kortum, “Automated image analysis of digital colposcopy for the detection of cervical neoplasia,” Journal of Biomedical Optics 13, p. 014029, 2008, incorporated herein by reference) designed a diagnostic image analysis framework using acetowhitening-based statistical features, reporting both patient-based and image-based diagnostic performances. The results showed 79% sensitivity and 88% specificity for the patient-based approach, and 82% sensitivity and 73% specificity for the image-based approach.
These currently reported diagnostic algorithms, however, have not fully taken advantage of cervical biology. Instead, the techniques have been generic rather than domain-specific, and have not been tailored to utilize the unique optical features of specific tissue, in this case, cervical tissue. The diagnostic performance of image analysis may be significantly enhanced by incorporating cervical cancer-specific (domain-specific) features in the algorithm design. For example, it is known that cervical cancer is mainly caused by the infection of metaplastic epithelium in the cervical transformation zone with one or more carcinogenic types of human papillomavirus (HPV) (D. A. Elson, R. R. Riley, A. Lacey, G. Thordarson, F. J. Talamantes, and J. M. Arbeit, “Sensitivity of the cervical transformation zone to estrogen-induced squamous carcinogenesis,” Cancer Research 60(5), p. 1267, 2000, incorporated herein by reference). In addition, some reports (see for example I. M. Orfanoudaki, G. C. Themelis, S. K. Sifakis, D. H. Fragouli, J. G. Panayiotides, E. M. Vazgiouraki, and E. E. Koumantakis, “A clinical study of optical biopsy of the uterine cervix using a multispectral imaging system,” Gynecologic Oncology 96(1), pp. 119-131, 2005; and J. Mirkovic, C. Lau, S. McGee, C. C. Yu, J. Nazemi, L. Galindo, V. Feng, T. Darragh, A. de Las Morenas, and C. Crum, “Effect of anatomy on spectroscopic detection of cervical dysplasia,” Journal of Biomedical Optics 14, p. 044021, 2009, incorporated herein by reference) have shown that differences in tissue structure between tissue types yield different optical properties for each tissue type. It has also been reported that the columnar tissue in the transformation zone of the cervix is spectroscopically distinct from the adjacent squamous tissue, and that these anatomical differences directly influence the spectroscopic diagnostic parameters (J. Mirkovic, C. Lau, S. McGee, C. C. Yu, J. Nazemi, L. Galindo, V. Feng, T. Darragh, A. de Las Morenas, and C. Crum, “Effect of anatomy on spectroscopic detection of cervical dysplasia,” Journal of Biomedical Optics 14, p. 044021, 2009, incorporated herein by reference). Furthermore, one study has shown that acetowhite response curves have different decays in the squamous tissue, the columnar tissue and the transformation zone (I. M. Orfanoudaki, G. C. Themelis, S. K. Sifakis, D. H. Fragouli, J. G. Panayiotides, E. M. Vazgiouraki, and E. E. Koumantakis, “A clinical study of optical biopsy of the uterine cervix using a multispectral imaging system,” Gynecologic Oncology 96(1), pp. 119-131, 2005, incorporated herein by reference). These studies and reports suggest that the performance of cervical cancer detection algorithms can be improved by incorporating tissue type information.
Previously reported methods also have not considered the spatial relationships between diagnostic features. For example, as taught in colposcopy textbooks (see B. S. Apgar, Brotzman, G. L. and Spitzer, M., “Colposcopy: Principles and Practice”, W.B. Saunders Company, Philadelphia, 2002, incorporated herein by reference), the presence of both acetowhitening and mosaicism indicates a high probability of cervical neoplasia (cervical cancer or an abnormal proliferation of cells in the cervix). These relationships between the features in a cervical image provide diagnostically valuable information in addition to that provided by the features themselves.
Most of the previous studies also lack image-based diagnostic performance assessment methods. Some of the studies used patient-based sensitivity and specificity. A patient-based performance analysis measures the algorithm's ability to accurately make positive and negative cancer predictions for each patient. However, patient-based performance analysis does not assess an algorithm's ability to correctly locate the image region containing cancerous tissue (abnormal area). If abnormal areas can be accurately located, the size of the surgical excision can be reduced, which in turn reduces the patient's discomfort. Likewise, accurate detection of the abnormal area can help pinpoint locations for biopsies. Therefore, a sound diagnostic performance measure for automated image analysis should assess an algorithm's ability not just to diagnose a patient, but also to locate the abnormal area.
The following patents and patent applications may be considered relevant to the field of the invention:
U.S. Pat. No. 6,236,881 to Zahler et al., incorporated herein by reference, discloses a computerized apparatus with a real time detection algorithm for non drug-activated imaging diseases, for example in cervical and bladder tissues.
U.S. Pat. No. 6,766,184 to Utzinger et al., incorporated herein by reference, discloses methods and apparatus for generating multispectral images of tissue. The multispectral images may be used as a diagnostic tool for conditions such as cervical cancer detection and diagnosis. Apparatus utilizing the invention include endoscopes and colposcopes.
U.S. Pat. No. 6,933,154 to Schomacker et al., incorporated herein by reference, provides methods for determining a characteristic of a tissue sample, such as a state of health, using spectral data and/or images obtained within an optimal period of time following the application of a chemical agent to the tissue sample.
U.S. Pat. No. 7,187,810 to Clune et al., incorporated herein by reference, provides methods of determining a correction for a misalignment between at least two images in a sequence of images due at least in part to sample movement. The methods are applied, for example, in the processing and analysis of a sequence of images of biological tissue in a diagnostic procedure. The invention also provides methods of validating the correction for a misalignment between at least two images in a sequence of images of a sample. The methods may be applied in deciding whether a correction for misalignment accurately accounts for sample motion.
U.S. Pat. No. 7,664,300 to Lange et al., incorporated herein by reference, discloses a uterine cervical cancer computer-aided-diagnosis (CAD) system consisting of a core processing system that automatically analyses data acquired from the uterine cervix and provides tissue and patient diagnosis, as well as adequacy of the examination.
U.S. Patent Application No. 2006/0039593 to Sammak et al., incorporated herein by reference, discloses methods and systems for determining characteristics of cellular structures. The methods include non-invasive, non-perturbing, automatable, and quantitative methods and may be applied to the examination of cells such as stem cells, embryos, and egg cells.
U.S. Patent Application Publication No. 2008/0039720 to Balas, incorporated herein by reference, discloses a quantitative method for determining tissue characteristics including the steps of generating data for a dynamic optical curve over a period of time based on an optical property of a tissue and determining a value of a dynamic optical parameter. The value of the dynamic optical parameter is compared with a reference value of the dynamic optical parameter known to be linked to a structural or functional characteristic and/or the pathological status of the tissue. Based on the comparison, a structural or functional characteristic and/or the pathological status of the tissue is determined. The method is used diagnose and/or grade neoplasia and/or HPV infection and/or calculating nuclear to cytoplasmic ratios of the cells in the tissue sample.
U.S. Patent Application Publication No. 2008/0101678 to Suliga et al., incorporated herein by reference, describes a Markov Random Field (MRF) based technique for performing clustering of images characterized by poor or limited data. The proposed method is a statistical classification model that labels image pixels based on the description of their statistical and contextual information. Apart from evaluating the pixel statistics that originate from the definition of the K-means clustering scheme, the model expands the analysis by the description of the spatial dependence between pixels and their labels (context), leading to the reduction of the inhomogeneity of the segmentation output with respect to the result of pure K-means clustering.
U.S. Patent Application Publication No. 2009/0034824 to Li et al., incorporated herein by reference, discloses a method for differentiating cancerous lesions from surrounding tissue, which includes extracting an opacity parameter from acetowhite regions of pre-acetic acid and post-acetic acid images of a cervix.
U.S. Patent Application Publication No. 2009/0253991 to Balas et al., incorporated herein by reference, discloses a method and an apparatus for the in vivo, non-invasive, early detection of alterations and mapping of the grade of these alterations, causing in the biochemical and/or in the functional characteristics of the epithelial tissues, during the development of tissue, atypias, dysplasias, neoplasias and cancers.
U.S. Patent Application Publication No. 2010/0027863 to Venkataraman et al., incorporated herein by reference, discloses a method for the detection of atypical vessels in digital cervical imagery.
U.S. Patent Application Publication No. 2010/0092064 to Li et al., incorporated herein by reference, discloses a rule-based unsupervised process for classifying cervical tissue by serially applying classifiers selected from the group comprising of determining size of texture region, opacity parameter, size of acetowhite regions, number of coarse and fine punctations, size of coarse and fine mosaics, size of atypical blood vessels and demographic data, so that the cervical tissue can be classified into no evidence of disease, low-grade dysplasia, high-grade dysplasia or cancer.